AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.04705	2.6
	2_DecisionTree	Decision Tree	logloss	0.388353	30.02
the best	3_Default_Xgboost	Xgboost	logloss	0.225148	62.51
	4_Default_NeuralNetwork	Neural Network	logloss	0.37337	10.48
	5_Default_RandomForest	Random Forest	logloss	0.353352	45.94
	Ensemble	Ensemble	logloss	0.225148	0.7

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

44.6 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.720978	0.902203	0.995327	0	0	0.833333	0.870705	0.575307	0.86804	0.353352
recall	0.83787	0.96295	0.995327	0	0	0.0458716	0.870705	0.47367	0.870705	0.353352
f1-score	0.775041	0.931587	0.995327	0	0	0.0869565	0.870705	0.464819	0.841842	0.353352
support	845	2807	428	5	12	327	0.870705	4424	4424	0.353352

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	708	137	0	0	0	0
Labeled as Emergency	102	2703	0	0	0	2
Labeled as Newborn	1	0	426	0	0	1
Labeled as Not Available	1	4	0	0	0	0
Labeled as Trauma	0	12	0	0	0	0
Labeled as Urgent	170	140	2	0	0	15

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Elective (Fold 1)

Dependence Emergency (Fold 1)

Dependence Newborn (Fold 1)

Dependence Not Available (Fold 1)

Dependence Trauma (Fold 1)

Dependence Urgent (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

61.2 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.822052	0.960354	0.990698	1	1	0.709677	0.92066	0.913797	0.918497	0.225148
recall	0.891124	0.966512	0.995327	0.8	0.0833333	0.538226	0.92066	0.712421	0.92066	0.225148
f1-score	0.855196	0.963423	0.993007	0.888889	0.153846	0.612174	0.92066	0.744423	0.917371	0.225148
support	845	2807	428	5	12	327	0.92066	4424	4424	0.225148

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	753	47	0	0	0	45
Labeled as Emergency	69	2713	0	0	0	25
Labeled as Newborn	0	0	426	0	0	2
Labeled as Not Available	1	0	0	4	0	0
Labeled as Trauma	0	11	0	0	1	0
Labeled as Urgent	93	54	4	0	0	176

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Elective (Fold 1)

Dependence Emergency (Fold 1)

Dependence Newborn (Fold 1)

Dependence Not Available (Fold 1)

Dependence Trauma (Fold 1)

Dependence Urgent (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

9.4 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.741701	0.938451	0.981087	0.5	0	0.490291	0.878391	0.608588	0.868829	0.37337
recall	0.846154	0.945137	0.969626	0.4	0	0.308869	0.878391	0.578298	0.878391	0.37337
f1-score	0.790492	0.941782	0.975323	0.444444	0	0.378987	0.878391	0.588505	0.871414	0.37337
support	845	2807	428	5	12	327	0.878391	4424	4424	0.37337

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	715	77	1	1	0	51
Labeled as Emergency	107	2653	1	1	0	45
Labeled as Newborn	1	4	415	0	0	8
Labeled as Not Available	1	2	0	2	0	0
Labeled as Trauma	0	11	0	0	0	1
Labeled as Urgent	140	80	6	0	0	101

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	1

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.822052	0.960354	0.990698	1	1	0.709677	0.92066	0.913797	0.918497	0.225148
recall	0.891124	0.966512	0.995327	0.8	0.0833333	0.538226	0.92066	0.712421	0.92066	0.225148
f1-score	0.855196	0.963423	0.993007	0.888889	0.153846	0.612174	0.92066	0.744423	0.917371	0.225148
support	845	2807	428	5	12	327	0.92066	4424	4424	0.225148

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	753	47	0	0	0	45
Labeled as Emergency	69	2713	0	0	0	25
Labeled as Newborn	0	0	426	0	0	2
Labeled as Not Available	1	0	0	4	0	0
Labeled as Trauma	0	11	0	0	1	0
Labeled as Urgent	93	54	4	0	0	176

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

28.8 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0.575	0.975571	0.990698	0	0	0.8125	0.846067	0.558961	0.884722	0.388353
recall	0.979882	0.882081	0.995327	0	0	0.0397554	0.846067	0.482841	0.846067	0.388353
f1-score	0.724726	0.926473	0.993007	0	0	0.0758017	0.846067	0.453335	0.827938	0.388353
support	845	2807	428	5	12	327	0.846067	4424	4424	0.388353

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	828	17	0	0	0	0
Labeled as Emergency	329	2476	0	0	0	2
Labeled as Newborn	1	0	426	0	0	1
Labeled as Not Available	1	4	0	0	0	0
Labeled as Trauma	1	11	0	0	0	0
Labeled as Urgent	280	30	4	0	0	13

Learning curves

Decision Tree

Tree #1

Rules

if (Emergency Department Indicator > 0.5) and (APR MDC Description <= 22.5) and (APR DRG Code > 3.5) then class: Emergency (proba: 98.3%) | based on 7,332 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight <= 250.0) and (APR Medical Surgical Description <= 0.5) then class: Elective (proba: 41.67%) | based on 2,119 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight <= 250.0) and (APR Medical Surgical Description > 0.5) then class: Elective (proba: 77.66%) | based on 2,055 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight > 250.0) and (CCS Diagnosis Code <= 219.5) then class: Newborn (proba: 99.38%) | based on 1,293 samples

if (Emergency Department Indicator > 0.5) and (APR MDC Description > 22.5) and (Facility Id <= 1739.0) then class: Emergency (proba: 84.88%) | based on 410 samples

if (Emergency Department Indicator > 0.5) and (APR MDC Description > 22.5) and (Facility Id > 1739.0) then class: Urgent (proba: 84.09%) | based on 44 samples

if (Emergency Department Indicator <= 0.5) and (Birth Weight > 250.0) and (CCS Diagnosis Code > 219.5) then class: Urgent (proba: 56.25%) | based on 16 samples

if (Emergency Department Indicator > 0.5) and (APR MDC Description <= 22.5) and (APR DRG Code <= 3.5) then class: Elective (proba: 50.0%) | based on 2 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Elective (Fold 1)

Dependence Emergency (Fold 1)

Dependence Newborn (Fold 1)

Dependence Not Available (Fold 1)

Dependence Trauma (Fold 1)

Dependence Urgent (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 6
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.9 seconds

Metric details

	Elective	Emergency	Newborn	Not Available	Trauma	Urgent	accuracy	macro avg	weighted avg	logloss
precision	0	0.634494	0	0	0	0	0.634494	0.105749	0.402582	1.04705
recall	0	1	0	0	0	0	0.634494	0.166667	0.634494	1.04705
f1-score	0	0.776379	0	0	0	0	0.634494	0.129397	0.492608	1.04705
support	845	2807	428	5	12	327	0.634494	4424	4424	1.04705

Confusion matrix

	Predicted as Elective	Predicted as Emergency	Predicted as Newborn	Predicted as Not Available	Predicted as Trauma	Predicted as Urgent
Labeled as Elective	0	845	0	0	0	0
Labeled as Emergency	0	2807	0	0	0	0
Labeled as Newborn	0	428	0	0	0	0
Labeled as Not Available	0	5	0	0	0	0
Labeled as Trauma	0	12	0	0	0	0
Labeled as Urgent	0	327	0	0	0	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back